Kernels from Probabilistic Models for Multiclass Classification and Reranking with Arbitrary Loss Function

Many classification problems involve loss functions different from the usual zero-one classification loss. In recent years, several approaches to accommodate loss functions in kernel-based learning algorithms have been suggested, but the construction of kernels has not been motivated by specific loss functions. We propose a method for deriving kernels from probabilistic models, which is tailored to a given loss function. We evaluate this method empirically using a natural language statistical parser as the probabilistic model, a SVM with slack rescaling as the learning algorithm for reranking of candidate parses provided by the statistical parser, and F1 measure over bracketed constituents as the loss function. The method with the proposed kernel achieves a significant improvement in F1 measure over the results with a kernel motivated by a zero-one loss function (a TOP kernel generalization for reranking) and over results of the statistical parser alone.